Optimization of variance-stabilizing transformations

Variance-stabilizing transformations are commonly exploited in order to make non-homoskedastic data easily tractable by standard methods. However, for the most common families of distributions (e.g., binomial, Poisson, etc.) exact stabilization is not possible and even achieving some approximate stabilization turns out to be rather challenging. We approach the variance stabilization problem as an explicit optimization problem and propose recursive procedures to minimize a nonlinear stabilization functional that measures the discrepancy between the standard deviation of the transformed variables and a xed desired constant. Further, we relax the typical requirement of monotonicity of the transformation and introduce optimized nonmonotone stabilizers which are nevertheless invertible in terms of expectations. We demonstrate a number of optimized variancestabilizing transformations for the most common distribution families. These stabilizers are shown to outperform the existing ones. In particular, optimized variance-stabilizing transformations for low-count Poisson, binomial, and negative-binomial data are presented.